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ABSTRACT: 

A multimedia system includes a multimedia device control program having a 
plurality of application program selectable controls for controlling operation 
of a like plurality of different classes of multimedia devices. Each class has 
a different set of controllable device functions or attributes. In response to 
selection of a class by an application program, a unique control panel is 
created in a user interface for controlling operation of a device. The 
selected control performs all of the processing actions necessary to control 
the device in response to user selections on the interface, without involvement 
of the application program in such actions. The multimedia control program 
includes a plurality of panel templates from which control panels are created 
in a control screen. Each panel has a plurality of controllers that can 
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manipulated by a user to control a corresponding function of a multimedia 
device. The templates provide a consistent interfaces having a common look and 
feel. The multimedia control program also includes a plurality of class 
control programs one or more of which are selected by an application program to 
be attached to a like number of multimedia devices for controlling operation of 
the devices in response to user selections on the user interface, thereby 
relieving the application program of performing the necessary control 
functions. 

20 Claims, 7 Drawing figures 

APD: 
19921215 

ABPL: 

A multimedia system includes a multimedia device control program having a 
plurality of application program selectable controls for controlling operation 
of a like plurality of different classes of multimedia devices. Each class has 
a different set of controllable device functions or attributes. In response to 
selection of a class by an application program, a unique control panel is 
created in a user interface for controlling operation of a device. The 
selected control performs all of the processing actions necessary to control 
the device in response to user selections on the interface, without involvement 
of the application program in such actions. The multimedia control program 
includes a plurality of panel templates from which control panels are created 
in a control screen. Each panel has a plurality of controllers that can 
manipulated by a user to control a corresponding function of a multimedia 
device. The templates provide a consistent interfaces having a common look and 
feel. The multimedia control program also includes a plurality of class 
control programs one or more of which are selected by an application program to 
be attached to a like number of multimedia devices for controlling operation of 
the devices in response to user selections on the user interface, thereby 
relieving the application program of performing the necessary control 
functions. 

BSPR: 

A still further object of the invention is to provide a multimedia device 
control program which creates a control panel with minimum involvement of an 
application program and which thereafter controls operation of the device 
without intervention of the application program. 

BSPR: 

Briefly, in accordance with the invention, a multimedia system includes a 
multimedia device control program having a plurality of application program 
selectable controls for a like plurality of different classes of multimedia 
devices. Each class has a different set of controllable device functions or 
attributes. In response to selection of a control by an application program, a 
unique control panel is created in a user interface for controlling operation 
of a device. The selected control performs all of the processing actions 
necessary to control the device in response to user selections on the 
interface, without involvement of the application program in such actions. 

BSPR; 

In accordance with another aspect of the invention, the multimedia application 
program includes a plurality of predetermined templates defining different 
control panels which contain user manipulate controllers. The application 
program sends a message to the control program indicating its selection and the 
control program creates the desired control screen from the selected template. 

DEPR: 

FIG. 2 illustrates an exemplary control screen 200 in which a plurality of 
windows or panels are displayed on a monitor in response to execution of a 



sample program named VIDPLAY". This program creates a display of a control 
screen 200 that includes a video window 202, an audio control panel 204, a 
video control panel 206, and a player control panel 208. Also presented in the 
screen is a standard cursor 210 that is manipulated through use of mouse 79. 
Window 202 contains a title bar 212 bearing the title "Sample VIDPLAY", a 
window control button 213, a sizing button 215, a menu bar 214, and a video 
viewing area 216. The window itself is conventional and can be manipulated by 
means of cursor 21 0 to change the position and size of the window. 

DEPR: 

Audio control panel 204 contains a title bar 218 identifying the panel as an 
"Audio" control panel, and a control area containing various simulated 
controllers or controls 220-224. Control 220 is a mute button, and controls 
221-224 are dials for respectively controlling volume, balance, treble, and 
bass. With reference to control 221 , each dial includes reference marks 228 
around the periphery of a circular knob, a movable pointer 229, a decrease 
button 226, and an increase button 227. Buttons 226 and 227 respectively 
display graphic symbols "-" and "+" which generally denote the button function. 
Also included in control 221 is a legend "Volume" signifying the function of 
the device being controlled. When the cursor 210 is placed on one of buttons 
and the mouse is clicked, the pointer appears to move until the mouse button is 
released. Such movement would be accompanied by signals being sent to the 
physical device whose controls are being simulated to cause the corresponding 
action to occur. For example, if the cursor is clicked on button 227, pointer 
229 rotates clockwise and the actual volume of the audio device would increase. 

DEPR: 

Video control panel 206 has a title bar 230 displaying the legend "Video", a 
freeze button 231 , and dial controls 232-236 for controlling the brightness, 
contrast, color, tint, and sharpness, of the video device, e.g., monitor 112 
(FIG. 1 ). Player control panel 208 displays the legend "Player Control" in 
title bar 240, push buttons 241-248, and a slider control 249. Buttons 241-248 
include different symbols symbolic of different player functions controlled 
thereby. Such buttons provide, for video disc player 1 08, controls of eject, 
pause, play, play backward, record, repeat, rewind, scan forward, scan 
backward, step backward (frame backward), step forward (frame advance), and 
stop. Slider control 249 displays the current device position in time or frame 
based units. 

DEPR: 

It is to be noted that the above discussion of FIG. 2 provides nomenclature 
facilitating an understanding of the invention as described below. The 
contents of control screen 200 as shown in FIG. 2 is intended to primarily show 
what samples of different control panels look like. Quite obviously, in any 
given system, the type and number of panels is dependent upon what multimedia 
devices are connected to the system, and what controls are selected to be 
shown. Further variations in such control panels and how they are created are 
described hereinafter. 

DEPR: 

FIG. 3 illustrates the layered structure of the software mechanism and the 
primary programs that are executed concurrently in a multitasking mode, in 
accordance with the invention. At the top, a user interface 126 communicates 
with MMAP 102 and MMDCP 106, both of which are above MMPM 104. The latter 
program interacts with the IBM OS/2 Presentation Manager (PM) program 99 which 
is above the operating system kernel 98, which sits on top of device drivers 
120. PM 99 also communicates with MMAP 102 and MMDCP 106. A device 
communications program 122 is between drivers 120 and BIOS 42 which is the 
lowest software layer and provides device specific processing for operating 
multimedia devices 124. These devices include video disc player 118, and any 
MMD 96 such as video tape, digital audio, compact disc (CD), and digital video 



devices. 



DEPR: 

MMAP 102 is a user oriented application program providing access to services of 
multimedia devices appropriate to the application, e.g., a video disc browsing 
program. MMAP 102 interacts with MMDCP 106 for the purpose of providing a user 
interface for the appropriate device, and with MMPM 104 for opening and closing 
the device and accessing specific features or performing custom processing not 
available via the more generic MMDCP 106. MMPM 104 implements a high level, 
device independent interface to multimedia devices for application use and 
includes support for device context switching, data streaming, and management 
of device contention. PM 99 is used for its screen management functions and 
capability of creating windows on a display screen. OS/2 kernel 98 represents 
the services provided by the IBM OS/2 operating system. These services 
generally include functions for process and resource management. Kernel 98 and 
PM 99 are omitted from later drawing figures for simplicity of illustration. 
OS/2 device drivers 120 are device specific control programs providing a 
software interface for controlling individual devices. A device driver at this 
level generally provides services for a device of a specific type and 
manufacturer. Device communications 122 indicates a communication mechanism by 
which a device driver accesses an attached device. Some devices are card type 
adapters installed in the system for access by the system bus. External 
devices are generally attached to the computer by a cable plugged into the 
parallel port or the serial port. 

DEPR: 

MMDCP 106 comprises an audio control 101, a video control 103, a player control 
105, one or more control instances 107. panel templates 109, interface logic 
111, and device logic 1 1 3. Controls 101,1 03, and 1 05 are separate generic 
routines or programs for controlling multimedia devices having audio, video, 
and player attributes. In general, an application program first opens a 
device, and then sends a message to MMDCP to create a control screen which is 
done by making an instance or copy of the appropriate control and then 
attaching the control instance to the device. That is, once the instance has 
been made, it is necessary to inform it of the identity of the device it will 
be controlling so that the commands can be sent to the proper device. After 
attachment, the control instance performs its primary function of controlling 
the particular device, as described in more detail hereinafter. 

DEPR: 

FIG. 4 illustrates various functions 1 34, 136, and 138 as performed in the 
prior art by a typical application program 102' and the interfaces between such 
program and user interface (Ul) 126' and MMPM 1 04. The illustration is for a 
system having a single multimedia device 124. The graphics on the screen 
display together with the means for manipulating the graphics and making 
selections, form the user interface 126' through which the user interacts with 
the system to control operation of the multimedia devices. Input messages 130 
from the user into the application program 102' include Ul selections made by 
the user actuating the various buttons, sliders, dials, etc. Output 
communications or messages 132 from the application to the user interface 
include controlling and updating the panel display, indicating changes in 
status of the multimedia devices, enabling and disabling buttons or controls 
which are or are not appropriate for the current device state. For example, if 
a device is not playing, then the pause button is disabled because its use is 
not applicable. Other application interactions 128, such as those not 
pertinent to the control of a MMD, may also occur with the user interface. 

DEPR: 

MMDCP 106 can create three different types of control panels: a video control 
panel for a device, such as a graphics/video overlay adapter, having video 
attributes; an audio control panel for a device, such as an amp/mixer device, 



having audio attributes; and a player control panel for a device, such as a 
videodisc player, having player attributes. A video control panel typically 
includes a freeze push button, and circular sliders or dials to control 
brightness, contrast, saturation, hue, and sharpness. An audio control panel 
typically includes a mute push button and a series of circular sliders to 
control volume, balance, treble, and bass. A player control panel typically 
includes push buttons for combinations of the following player actions: eject, 
pause, play backward,play forward, play reverse, record, repeat, rewind, scan 
backward, scan forward, scan reverse, step backward, step forward, and stop. A 
player control panel might also include a linear slider for position display. 
It should be obvious that the individual controls depend on the capabilities of 
the attached multimedia device(s) and the configuration of the player control 
screen. 

DEPR: 

FIG. 6 illustrates the manner in which MMDCP 106 controls two multimedia 
operations for a MMAP that runs two multimedia devices. The devices might be , 
e.g., a player device 124-1 such as videodisc player 1 08 connected to the 
serial port, and an audio device 124-2 connected to one of the expansion 
connectors 50 and accessible over the system bus. For such a configuration, 
MMAP 102 needs to send a first DEVICE OPEN message 162-1, a first DEVICE ID 
message 152-1, and a first CREATE message 150-1 to create a player control 
instance 107P, and to then send a second DEVICE OPEN message 162-2, a second 
DEVICE ID message 152-2, and a second CREATE message 150-2 to create an audio 
control instance 107A. Also, respective player and audio device drivers 120-1 
and 120-2 and respective device communications 122-1 and 122-2 have to have 
been loaded into the system, for establishing communications with the 
respective multimedia devices. With two devices, the control screen 200 in 
FIG. 2 could display player control panel 208 and audio control panel 204. 
Once the control instances are attached, each control instance manages its 
attached device and configures itself according to the device capabilities. 
During operation, player control instance 107P receives communications 130-1 
for the Ul and sends messages 132-1 to the Ul, while audio control instance 
107A receives messages 130-2 from the Ul and sends messages 132-2 to the Ul. 
In the prior art, the application program has to implement and manage the 
additional functionality to address and handle the functional differences 
between player and audio devices. 

DEPR: 

FIG. 7 shows further details of control message processing that is done in 
response to receiving messages or inputs 1 70, 1 72, 1 74, and 1 76 from four 
sources, namely, MMAP 102, Ul 126, the operating system, and multimedia device 
124 inputs through MMPM 104. In response to receiving a CREATE control screen 
command, MMDCP is operative to perform the functions 1 78 and acquire the 
resources (e.g., memory) for the display, make the desired control panels, and 
place them on the screen. In response to receiving a DESTROY message from the 
application, actions 180 release the resources, and discard the window and any 
control panels on the screen. In response to receipt of a DEVICE ID message 
from the application, actions 1 82 comprise checking the validity of the device 
ID, making a control instance by copying the control and then attaching the 
control instance to the open device, determining device capabilities, querying 
the device status, and updating the control panels, e.g., by enabling or 
disabling components and changing current settings. The attaching is simply 
notifying the control instance of the ID of device to be controlled. Actions 
184 are functions or actions performed in response to miscellaneous messages 
from the application and include enabling or disabling the sending of 
notification messages to the application, toggling application notify on/off, 
toggling device communication on/off, and using custom control panel layout. 



DEPR: 

Action 186 updates control panels as necessary in response to receiving 



messages from the operating system. Such messages include hide, show, move, 
paint, etc. In response to receiving device notification messages, actions 188 
include updating the control panel as necessary. If an error has occurred, a 
recovery atfempt is made or the component is disabled. Timed events are events 
which occur at a device asynchronously with respect to a command issued to the 
device. For example, if the device is instructed to play for a certain amount 
of time, the completion of the play command is a timed event which results in a 
notification being sent to MMDCP 106. At this point, MMDCP 106 stops animating 
the play button and changes the appearance of the play button to be consistent 
with a "stop" state. Other timed events might include position notifications 
and cuepoints. When an input 172 is received, actions 190 are performed to 
check the current control panel/device context, and determine what actions are 
necessary. If for example, a device is already playing and the play button is 
selected, the context indicates no action is necessary and the selection of the 
play button is ignored. The actions to be done include cancelling current 
operations, sending device commands, examining device command responses, 
attempting error recovery, and updating the control panel as necessary. The 
specific details of the actions described relative to FIG. 7 should be obvious 
to those skilled in the art and have been omitted to shorten description 
particularly since the details are not claimed. 

DEPR: 

A typical sequence of actions that occur can be illustrated by the example of 
audio control panel 204 and volume controller 221 (FIG. 2). Suppose the user 
desires a volume increase. To accomplish this using the mouse, the mouse is 
manipulated to position cursor 210 on button 227 and then clicked and held 
until the volume increases to the desired level. In response to the mouse 
being clicked, the following sequence of actions occur. First, presentation 
manager 99 (FIG. 3) determines which application, the cursor is currently being 
operated under, which window the cursor is being operated in, and which object 
in the window has been selected. In this example, the object is the audio 
controller volume increase button. PM 99 then sends a user interface message 
172 (FIG. 7) to the audio control instance which message identifies the object 
and the control action desired. In response to receiving the message, the 
audio control instance then performs the functions 190 and determines what 
actions are necessary. The first is to decide whether to cancel any current 
operation. If the mute button is on, it would be cancelled to allow audio to 
be produced by the device. Next, a volume increase device command is sent to 
the audio device to increase the volume by an increment. At this point the 
device responds and if the response indicates such action has been done without 
error, the next action is to update the pointer of the volume dial and this is 
done by sending a message through PM 99 and user interface which thereupon 
redraws the volume dial pointer to a new position representing an increment of 
volume increase. The sequence of actions repeat until the mouse button is 
released, or until the maximum volume is reached. An optional action (not 
shown) could be performed before communicating with the device, to give the 
application program the opportunity to handle the user selection. This is 
useful in situations where operational customization is done by the 
application, instead of operating by the control instance. With this option, 
the application would send a return code indicating it will or will not handle 
the message. 

DEPR: 

As indicated above, MMDCP 106 includes a variety of panel templates 109 that 
are used in making a control panel in 178. MMDCP provides the application 
programmer with the options of accepting a default panel, a minimal panel, a 
complete panel, or a custom panel. A complete panel includes all the supported 
dials, buttons, sliders, etc. for controlling a particular device. A minimal 
panel includes only those controllers generally considered absolutely 
necessary. Naturally, the minimal panel occupies less screen area than a 
complete panel. A custom panel could be defined by the application and use any 
subset of the components of a complete panel, change relative positions, 





include other functions, etc. The templates also include vertically oriented 
panels, such as the audio and video panels shown in FIG. 2, and horizontally 
oriented panels, for both complete and minimal panels. Additional options 
include animated buttons (such as movement of arrows on a Play button when a 
device is playing), and the ability for an application to request that the 
control only present a user interface but not communicate with the device. 
This option may be useful if the application desires to control other devices 
while taking advantage of the user interface standards and processing. The 
designs of the panels provide a uniform appearance and common look and feel 
between different applications. 

DEPR: 

In response to successful execution of the above code, a device ID is obtained, 
a window is created, a player control instance is created, and the device is 
set. The player control instance will display the control screen including the 
player control panel, handle all user interface actions, communicate with the 
videodisc player, etc. No other code in the application program is needed to 
perform or process selections from the control panel . 

DEPR: 

The window style of the MMDCP specifies, among other things, the orientation of 
the control panel (vertical or horizontal) and the degree of desired device 
control (minimal, complete). The control program uses the style to select one 
of four layouts for the initial appearance. For example, audio control 204 in 
FIG. 2 is a vertical, complete control. The result thus promotes a consistent 
user interface. 



The panel templates 1 09 each specify the type of components and their positions 
using a dialog template of PM. By way of example, the dialog template 
corresponding to the layout of audio control panel 204 is: 

DEPR: 

For the purpose of defining the layout of a control panel, the window class 
specified in the dialog template is ignored for each of the predefined 
components such as the mute button or the volume dial. To create the control 
panel display, the MMDCP uses the information in the template to create each of 
the desired components. The subset of components present influences the 
subsequent operation of the MMDCP. For example, a minimal audio control 
includes only a volume dial and a mute button. A request to update the control 
panel would query the device only for the applicable information, i.e., the 
volume and mute status. A complete audio control would cause a query of the 
device for all of the components, i.e., the mute, volume, bass, and treble 
status. 

DEPR: 

The template mechanism also allows the MMAP to specify an alternative layout 

for a control panel . This is done through an optional miscellaneous 

application message 184 to USE CUSTOM CONTROL PANEL LAYOUT. In this case, the 

MMAP provides the desired template to the MMDCP after the panel is created. 

The custom template supersedes any layout selected at MMDCP creation time. If 

neither minimal nor complete is specified in the style, then the MMDCP assumes 

that the MMAP will subsequently provide a layout and no components are created 

until this is done. 



//Open a videodisc device with an alias name and obtain a //device ID 
OpenDevice ((LPSTR)szVdiscDevName, (LPSTR)szVdiscAlias, &wVdiscDevlD); 
//Create a window to hold the Player Control Panel on the //screen hwndVPCDIg 
= WinLoadDlg(HWND.sub.- DESKTOP, hwndClient, pVPCDIgProc, 0, ID.sub.- 



DEPR: 



DETL: 



******** 



BEGIN EXAMPLE CODE 



DLG.sub- VPC, NULL); //Create the Raver Control Panel using MINIMAL 
//configuration. "hwndVPC" is the handle uses for any further //communication 
with the player control hwndVPC = WinCreateWindow (hwndVPCDig, //Use the 
window above WC.sub - PLAYERCTL, //Create Player Control " " //Misc OS/2 
item WS.sub.- VISIBLE .vertline. MCS.sub.- MINIMAL //Select panel style 
X.sub.- MMCTL, Y.sub.- MMCTL //Starting screen location 200,1 00 //Size 
hwndVPCDig, //Misc OS/2 item HWND.sub.- TOP //Misc OS/2 item ID.sub.- 
VPC.sub.-WND, //Misc OS/2 item NULL, //Misc OS/2 item NULL); //Misc OS/2 
item //Inform the Player Control to use the videodisc player that //was 
opened earlier WinSendMsg (hwndVPC, MCM.sub.- SETDEVICEALIAS, szVdiscAlias, 
0) *"*****END SAMPLE CODE ******** 

CLPR: 

7. A MMDPS in accordance with claim 1 wherein said MMDCM further comprises a 
plurality of predefined control panels for each class of said controls, said 
MMDCM being further operative in response to said CREATE message to select one 
of said predefined control panels for display in said control screen. 

CLPR: 

8. A MMDPS in accordance with claim 7 wherein said predefined control panels 
for each class include a minimal control panel and a complete control panel. 
said minimal control panel having a minimum number of controllers for 
controlling a minimum number of functions, and said complete control panel 
having a plurality of controllers for controlling a maximum number of functions 

of each class. 

CLPR: 

12. A MMDPS in accordance with claim 1 1 wherein said minimal control panel 
templates and said complete control panel templates further define control 
panels of horizontal and vertical orientations. 

CLPV: 

said selected control comprising third means for creating a control screen on 
said monitor in said user interface, said control screen having a control panel 
comprising a plurality of user actuated controllers for controlling different 
functions of said open MMD; 

CLPV: 

said MMDCM further comprises a plurality of predefined control panels for each 
class of said controls, said MMDCM being further operative in response to said 
CREATE message to select one of said predefined control panels for display in 
said control screen; 

CLPV: 

and said processing means is further operative in response to receiving said 
DEVICE ID message to determine device capabilities and device status, and send 
messages to said user interface for updating said control panel in accordance 
with said device capabilities and said device status. 

CLPV: 

said MMDCM is further operative in response to receiving a second CREATE 
message to create a second control instance and a second control panel for a 
second MMD. 

CLPV: 

said predefined control panels further comprise control panels of horizontal 
and vertical orientations. 

CLPV: 

and multimedia device control means (MMDCM) layered between said first means 
and said second means, said MMDCM comprising a plurality of control panel 



templates for each class of said controls, said templates for each class 
include a minimal control panel template and a complete control panel template, 
said minimal control panel template defining a minimum number of controllers 
for controlling a minimum number of functions, and said complete control panel 
template defining a plurality of controllers for controlling a maximum number 
of functions of each class; 

CLPV: 

said MMDCM being responsive to a CREATE message from the application program to 
select one of said templates and send a message to said user interface to 
create a control panel in said control screen in accordance with the definition 
in the template so selected; 

CLPV: 

sa 'd control panel in said control screen comprising a plurality of user 
actuated controllers for controlling different functions of said open MMD; 

CLPV: 

(d) creating a control screen on said monitor in said user interface, said 
control screen including a control panel comprising a plurality of user 
actuated controllers for controlling different functions of said open MMD; 

CLPV: 

storing in said storage system a plurality of control panel templates for each 
class of said controls, said templates for each class include a minimal control 
panel template and a complete control panel template, said minimal control 
panel template defining a minimum number of controllers for controlling a 
minimum number of functions, and said complete control panel template defining 
a plurality of controllers for controlling a maximum number of functions of 
each class; 

CLPV: 

said step (d) being performed by creating said control panel in said control 
screen in accordance with the definition in the one template so selected. 

CLPV: 

determining device capabilities and device status, and sending a message from 
said control instance to said user interface to update said control panel in 
accordance with said device capabilities and said device status. 

CLPV: 

storing in said storage system a plurality of control panel templates for each 
class of said controls, said templates for each class include a minimal control 
panel template and a complete control panel template, said minimal control 
panel template defining a minimum number of controllers for controlling a 
minimum number of functions, and said complete control panel template defining 
a plurality of controllers for controlling a maximum number of functions of 
each class; 

CLPV: 

including in said application program a template defining a custom control 
panel : 

CLPV: 

said step (d) creates a custom control panel in said control screen when said 
custom control panel template is specified in said CREATE message. 



